-
Notifications
You must be signed in to change notification settings - Fork 8.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
HADOOP-18357. Retarget solution file to VS2019 #4616
Conversation
* This PR retargets the Visual Studio solution and vcxproj files to use the latest version of Visual Studio (2019 as of this writing).
This reverts commit 8093b32.
🎊 +1 overall
This message was automatically generated. |
@@ -39,12 +37,14 @@ | |||
<UseDebugLibraries>false</UseDebugLibraries> | |||
<WholeProgramOptimization>true</WholeProgramOptimization> | |||
<CharacterSet>Unicode</CharacterSet> | |||
<PlatformToolset>v142</PlatformToolset> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What was the old number and where was it defined?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the old number should've been v100
. I'm not sure why it wasn't defined. Visual Studio automatically added this when I ran the command to retarget the project files.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can we check if it was inherited from somewhere else?
This may trigger inconsistencies.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So how does it know which VS version to be scoped to?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I ran the retarget command from VS2019 command prompt
, thus the retarget tool added the PlatformToolset version corresponding to VS2019.
The retarget command would take care of inserting the right version based on Visual Studio command prompt that we choose. Please also note that the subsequent mvn install
command must be run from the same command prompt.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given that we didn't have the target before; why would we add it now?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why would we add it now?
The compilation fails without the PlatformToolset
when compiled with Visual Studio 2019 (I'm not sure about VS2010). I'm fine with documenting running the retarget command as a pre-requisite step prior to running the mvn install
command, if we don't want to add the PlatformToolset
here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Take a look at HADOOP-14667
Specially: MSBuild Solution files are converted to the version of VS at build time
If this makes sense, we go with this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is amazing. Thanks for sharing this @goiri 😊. I'm able to confirm that while building the hadoop-common-project
module, the vcxproj and solution files automatically get upgraded to the appropriate Visual Studio version. All thanks to
hadoop/hadoop-common-project/hadoop-common/pom.xml
Lines 844 to 923 in 95a8587
<plugin> | |
<groupId>org.codehaus.mojo</groupId> | |
<artifactId>exec-maven-plugin</artifactId> | |
<executions> | |
<execution> | |
<id>convert-ms-winutils</id> | |
<phase>generate-sources</phase> | |
<goals> | |
<goal>exec</goal> | |
</goals> | |
<configuration> | |
<executable>${basedir}\..\..\dev-support\bin\win-vs-upgrade.cmd</executable> | |
<arguments> | |
<argument>${basedir}\src\main\winutils</argument> | |
<argument>${project.build.directory}</argument> | |
</arguments> | |
</configuration> | |
</execution> | |
<execution> | |
<id>compile-ms-winutils</id> | |
<phase>compile</phase> | |
<goals> | |
<goal>exec</goal> | |
</goals> | |
<configuration> | |
<executable>msbuild</executable> | |
<arguments> | |
<argument>${basedir}/src/main/winutils/winutils.sln</argument> | |
<argument>/nologo</argument> | |
<argument>/p:Configuration=Release</argument> | |
<argument>/p:OutDir=${project.build.directory}/bin/</argument> | |
<argument>/p:IntermediateOutputPath=${project.build.directory}/winutils/</argument> | |
<argument>/p:WsceConfigDir=${wsce.config.dir}</argument> | |
<argument>/p:WsceConfigFile=${wsce.config.file}</argument> | |
</arguments> | |
</configuration> | |
</execution> | |
<execution> | |
<id>convert-ms-native-dll</id> | |
<phase>generate-sources</phase> | |
<goals> | |
<goal>exec</goal> | |
</goals> | |
<configuration> | |
<executable>${basedir}\..\..\dev-support\bin\win-vs-upgrade.cmd</executable> | |
<arguments> | |
<argument>${basedir}\src\main\native</argument> | |
<argument>${project.build.directory}</argument> | |
</arguments> | |
</configuration> | |
</execution> | |
<execution> | |
<id>compile-ms-native-dll</id> | |
<phase>compile</phase> | |
<goals> | |
<goal>exec</goal> | |
</goals> | |
<configuration> | |
<executable>msbuild</executable> | |
<arguments> | |
<argument>${basedir}/src/main/native/native.sln</argument> | |
<argument>/nologo</argument> | |
<argument>/p:Configuration=Release</argument> | |
<argument>/p:OutDir=${project.build.directory}/bin/</argument> | |
<argument>/p:CustomZstdPrefix=${zstd.prefix}</argument> | |
<argument>/p:CustomZstdLib=${zstd.lib}</argument> | |
<argument>/p:CustomZstdInclude=${zstd.include}</argument> | |
<argument>/p:RequireZstd=${require.zstd}</argument> | |
<argument>/p:CustomOpensslPrefix=${openssl.prefix}</argument> | |
<argument>/p:CustomOpensslLib=${openssl.lib}</argument> | |
<argument>/p:CustomOpensslInclude=${openssl.include}</argument> | |
<argument>/p:RequireOpenssl=${require.openssl}</argument> | |
<argument>/p:RequireIsal=${require.isal}</argument> | |
<argument>/p:CustomIsalPrefix=${isal.prefix}</argument> | |
<argument>/p:CustomIsalLib=${isal.lib}</argument> | |
</arguments> | |
</configuration> | |
</execution> | |
</executions> | |
</plugin> |
[INFO] --- exec-maven-plugin:1.3.1:exec (convert-ms-native-dll) @ hadoop-common ---
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\devenv.com
C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\Common7\IDE\devenv.exe
Microsoft Visual Studio 2019 Version 16.11.9.
Copyright (C) Microsoft Corp. All rights reserved.
Upgrading project 'native'...
Configuration 'Release|Win32': changing Platform Toolset to 'v142' (was 'v100').
Configuration 'Release|x64': changing Platform Toolset to 'v142' (was 'v100').
Migration completed successfully, but some warnings were detected during migration.
For more information, see the migration report:
E:\hadoop-common-project\hadoop-common\src\main\native\UpgradeLog.htm
I'm able to fully build the trunk as well -
I'll abandon this PR since I'm able to build the trunk on Windows without these changes.
hadoop-common-project/hadoop-common/src/main/native/native.vcxproj
Outdated
Show resolved
Hide resolved
hadoop-common-project/hadoop-common/src/main/native/native.vcxproj.filters
Show resolved
Hide resolved
🎊 +1 overall
This message was automatically generated. |
The vcxproj and solution files automatically get retargeted due to - hadoop/hadoop-common-project/hadoop-common/pom.xml Lines 844 to 923 in 95a8587
I'm able to build the trunk on Windows without this PR - Closing this PR since these changes are not needed. |
Description of PR
The Visual Studio version used by
winutils
andnative
components in Hadoop common are quite old. We need to retarget the solution and vcxproj files to use the latest version (Visual Studio 2019 as of this writing).With this PR, we can finally build Hadoop on Windows without any hacks.
Hadoop tar
Hadoop distribution
Hadoop native libraries
How was this patch tested?
For code changes:
LICENSE
,LICENSE-binary
,NOTICE-binary
files?